Users access the Internet today using devices ranging from handhelds to powerful workstations, over connections ranging from 56 Kbps modems to high speed 100 Mb/s Ethernet. Even though the available bandwidth, display and processing capabilities continue to grow, the heterogeneity and the spread of capabilities at any point in time are here to stay. On the other hand, as bandwidth and other factors grow, so does the richness of media that is delivered to users.
Under these circumstances, a rigid media representation format, producing decompressed content only at a fixed resolution and quality is clearly inappropriate. A delivery system based on such a compression scheme can only deliver content satisfactorily to a small subset of users interested in the content. The rest, either cannot receive anything at all, or receives poor quality and/or resolution relative to the capabilities of their network connections and/or accessing devices. The inability to cater to this diversity has been a determining factor that stunted the growth of new rich media, because such rich content can cater only to power users comprising a small fraction of the whole. Without adequate focus on seamless content adaptation, accessibility and usability of media will always be severely limited.
One known technique for providing media content to users having a variety of capabilities and preferences is to provide multiple versions of the media suiting a variety of capabilities and preferences. While this approach works with delivery models where the recipient directly connects to the media originator, for any other multi-hop, multi-recipient delivery scenario, too much redundancy and inefficiency is introduced, leading to wastage of bandwidth and storage. This is especially so, when the media creator wishes to provide a wide range of choices catering to a large consumer base, and therefore needs to maintain a large number of versions differing in a variety of ways.
In order to combat the above problem, scalable compression formats have been proposed. Scalable compressed representations can accommodate all users by automatically maximizing the multimedia experience for a given user's computing power and connection speed. By adapting rich media content written for high-end machines with fast connections, to less powerful machines with slower connections, the overheads involved in producing different versions as described above for different scenarios can be virtually eliminated. Furthermore, content created today at the highest possible quality, remains ‘timeless’ when represented in a scalable format, and the experience it provides gradually increases, as the power of machines and connection speeds improve.
One example of a scalable compressed representation is JPEG2000. JPEG2000 is a scalable standard for still images and endeavors to combine quality scalability and resolution scalability in a format specific to JPEG2000 compressed data, to enable distribution and viewing over a variety of connections and devices. However, in order to obtain the benefits of the scalability of the format, it is necessary to develop and deploy an infrastructure that specifically supports transcoding of JPEG2000 content and delivery to a heterogeneous recipient base.
In recent years, a great deal of attention has been focused on delivering streaming video over the Internet or wireless. Hence, video standards of MPEG-X (mostly MPEG-4) and H.26X families were developed that incorporate various forms of scalability for delivering media content such as streaming video to a heterogeneous recipient base. However, this type of scalable video over the Internet is limited to maintaining multiple versions for a few different types of connections, because complete infrastructures that support transport of scalable video formats are non-existent.
Any infrastructure, is expensive to deploy, and requires significant financial commitments from the patron companies or patron consortia. In order to guarantee constancy of the format it would also be desirable that the format the content is represented in be standardized. On the other hand, standards take several years to come into effect, typically much longer than is commensurate with the normal pace of change in the multimedia industry. As new types of media beyond traditional images, video and audio evolve; it would become more and more difficult to expect standards to support their representation. Even if scalable formats evolved for every new type of media, the inevitable difference in the structure of the content would necessitate use of different infrastructures for scalable delivery of different types of media. The expenses involved present a very formidable obstacle in adoption of such new media and supportability of its scalability features.
There are various types of bit-stream scalability that can be devised depending on the type of media. For example, SNR (quality) scalability refers to progressively increasing quality as more and more of the bit-stream is included, and applies to most types of media. Resolution scalability refers to fineness of spatial data sampling, and applies to visual media such as images, video, 3D etc. Temporal scalability refers to fineness of sampling in the time-domain, and applies to video and other image sequences. There are several types of scalability pertaining to audio, such as number of channels and sampling frequency and so on. In the future, with the evolution of newer, richer and more interactive types of media, there will be newer types of scalability that are, to date, unknown. A scalable bit-stream does not always have a single type of scalability. Different types of scalability can co-exist, so as to provide a range of adaptation choices.
In new rich media, different media elements are often bundled together to provide a composite media experience. According to one known technology, an image with audio annotation and some animation provides a composite experience of a presentation using three media elements (an image, an audio clip, some animation data). Composite rich media models such as this lead to newer types of scalability specific to the media, because certain non-critical elements of the composite may be dropped to accommodate other more critical ones within the limited resources of a recipient.
The present invention provides a system, method, and format thereof for scalable encoded media data delivery to a heterogeneous recipient base and that is not specific to the type of media being delivered thereby requiring a single delivery infrastructure for delivering all types of currently known media and future media.